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ABSTRACT 

In  this  paper,  a  framework  for  recognizing  network  traffic  in  order  to  detect  anomalies  is  proposed. 
We  propose  to  combine  and  correlate  parameters  from  different  layers  in  order  to  detect  O-day  attacks 
and  reduce  False  Positives.  Moreover,  we  propose  to  combine  statistical  and  signal-based  features.  The 
major  contribution  of  this  paper  are:  novel  framework  for  network  security  based  on  the  correlation 
approach  as  well  as  new  signal  based  algorithm  for  intrusion  detection  using  Matching  Pursuit. 


1.0  INTRODUCTION  AND  MOTIVATION 

Intrusion  Detection  Systems  (IDS)  are  based  on  mathematical  models,  algorithms  and  architectural 
solutions  proposed  for  correctly  detecting  inappropriate,  incorrect  or  anomalous  activity  within 
a  networked  systems.  Intrusion  Detection  Systems  can  be  classified  as  belonging  to  two  main  groups 
depending  on  the  detection  technique  employed:  anomaly  detection  and  signature -based  detection. 
Anomaly  detection  techniques,  that  we  focus  on  in  our  work,  rely  on  the  existence  of  a  reliable 
characterization  of  what  is  normal  and  what  is  not,  in  a  particular  networking  scenario.  More  precisely, 
anomaly  detection  techniques  base  their  evaluations  on  a  model  of  what  is  normal,  and  classify 
as  anomalous  all  the  events  that  fall  outside  such  a  model.  If  an  anomalous  behaviour  is  recognized,  this 
does  not  necessarily  imply  that  an  attack  activity  has  occurred:  only  few  anomalies  can  be  actually 
classified  as  attempts  to  compromise  the  security  of  the  system. 

Anomaly  Detection  Systems  can  be  classified  according  to: 

•  the  used  algorithm, 

•  analyzed  features  of  each  packet  singularly  or  of  the  whole  connection, 

•  the  kind  of  analyzed  data  -  whether  they  focus  on  the  packet  headers  or  on  the  payload. 

Most  current  IDS  systems  have  problems  in  recognizing  new  attacks  (O-day  exploits)  since  they  are  based 
on  the  signature -based  approach.  In  such  mode,  when  system  does  not  have  an  attack  signature  in 
database,  such  attack  is  not  recognized.  Another  drawback  of  current  IDS  systems  is  that  the  used 
parameters  and  features  do  not  contain  all  the  necessary  information  about  traffic  and  events  in  the 
network. 

Therefore,  in  this  paper  we  present  the  framework  in  which  anomaly  detection  system  based  on  correlation 
and  diversity  approaches  are  used,  such  as: 

•  item  diversity  -  different  network  layers  parameters  are  monitored  and  used.  In  such  approach 
we  do  not  have  information  from  transport  layer  only  -  such  information  is  merged/correlated 
with  application  layer  events. 
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•  correlation  -  correlation  is  used  twofold  (during  decision): 

•  item  both  anomaly  and  signature -based  approaches  are  correlated, 

•  parameters/features  from  various  network  layers  are  correlated, 

•  statistical  and  signal-based  features  are  used  and  correlated. 

2.0  TECHNICAL  SOLUTION 

In  this  paper,  a  new  solution  for  ADS  system  based  on  signal  processing  algorithm  is  presented.  ADS 
analyzes  traffic  from  internet  connection  in  certain  point  of  a  computer  network.  The  proposed  ADS 
system  uses  redundant  signal  decomposition  method  based  on  Matching  Pursuit  algorithm.  ADS  based  on 
Matching  Pursuit  uses  Dictionary  of  Base  Functions  -  BFD  to  decompose  input  ID  traffic  signal  (ID 
signal  may  represent  packets  per  second)  into  set  of  based  functions  called  also  atoms.  The  proposed  BFD 
has  a  ability  to  approximate  traffic  signal.  Number  and  parameters  of  base  functions  was  limited  in  order 
to  shorten  atom  search  time  process 

Since  some  attacks  are  visible  only  in  specific  layer  (e.g.  SQLIA),  in  our  approach,  we  propose  to  use 
network  parameters  from  different  layers. 

Transport  layer,  network  layer  and  application  layer  parameters  are  used 

In  the  further  step,  we  use  the  presented  parameters  to  calculate  characteristics  (features)  of  the  observed 
traffic.  Some  of  the  parameters  are  used  for  statistical  features  calculation  and/or  for  signal-based  feature 
calculation  respectively.  Feature  extraction  methods  are  presented  in  the  following  subsections 


2.1  Statistical  Features 

The  Chi-Square  multivariate  test  for  Anomaly  Detection  Systems  can  be  represented  by  equation  1 : 

f  (Xj-Xj3 

(i) 


Where  ^  )  denote  an  observation  of  V  variables  from  a  process  at  time  ^  and 

£X  =  f  x]L  x2 ...  Jl  )  .  u 

V.  J  '  p  /  is  the  sample  mean  vector. 


Using  only  the  mean  vector  in  Equation  (1),  cause  that  Chi-Square  multivariate  test  detects  only  the  mean 
shift  on  one  or  more  of  the  variables. 


2.2  Signal  Processing  Features 

Signal  processing  techniques  have  found  application  in  Network  Intrusion  Detection  Systems  because  of 
their  ability  to  detect  novel  intrusions  and  attacks,  which  cannot  be  achieved  by  signature -based 
approaches.  It  has  been  shown  that  network  traffic  presents  several  relevant  statistical  properties  when 
analyzed  at  different  levels  (e.g.  self-similarity,  long  range  dependence,  entropy  variations,  etc.) 

Approaches  based  on  signal  processing  and  on  statistical  analysis  can  be  powerful  in  decomposing  the 
signals  related  to  network  traffic,  giving  the  ability  to  distinguish  between  trends,  noise,  and  actual 
anomalous  events.  Wavelet-based  approaches,  maximum  entropy  estimation,  principal  component 
analysis  techniques,  and  spectral  analysis,  are  examples  in  this  regard  which  have  been  investigated  in  the 
recent  years  by  the  research  community.  However,  Discrete  Wavelet  Transform  provides  a  large  amount 
of  coefficients  which  not  necessarily  reflect  required  features  of  the  network  signals. 
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Therefore,  in  this  paper  we  propose  another  signal  processing  and  decomposition  method  for 
anomaly/intrusion  detection  in  networked  systems.  We  developed  original  Anomaly  Detection  Type  IDS 
algorithm  based  on  Matching  Pursuit. 

In  the  rest  of  the  paper,  our  original  ADS  method  will  be  presented  in  details.  Moreover,  results 
of  experimental  setup  will  be  given.  We  tested  our  method  with  standard  traces  in  Worm  detection 
scenario  as  well  as  in  anomaly  detection  scenario.  Discussion  on  redundant  dictionary  parameters  and  final 
conclusions  will  be  provided. 

2.2.1  Matching  Pursuit 

Matching  Pursuit  signal  decomposition  was  proposed  by  Mallat  and  Zhang. 

Matching  Pursuit  is  a  greedy  algorithm  that  decomposes  any  signal  into  a  linear  expansion  of  waveforms 
which  are  taken  from  an  over  complete  dictionary  D.  The  dictionary  D  is  an  over  complete  set  of  base 
functions  called  also  atoms. 


D  =  {ay  :  y  e  T} 


(2) 


where  every  atom  a  from  dictionary  has  norm  equal  to  1 : 


r  represents  set  of  indexes  for  atom  transformation  parameters  such  as  translation,  rotation  and  scaling. 

Signal  s  has  various  representations  for  dictionary  D  Signal  can  be  approximated  by  set  of  atoms  ak  from 
dictionary  and  projection  coefficients  ck  : 


s  = 


|d|-i 

2lckak 


n= 0 


(3) 


To  achieve  best  sparse  decomposition  of  signal  s  (min)  we  have  to  find  vector  ck  with  minimal  norm 
but  sufficient  for  proper  signal  reconstruction.  Matching  Pursuit  is  a  greedy  algorithm  that  iteratively 
approximates  signal  to  achieve  good  sparse  signal  decomposition.  Matching  Pursuit  finds  set  of  atoms  a 

such  that  projection  of  coefficients  is  maximal.  At  first  step,  residual  R  is  equal  to  the  entire  signal 
R0  =  s . 


(4) 


If  we  want  to  minimize  energy  of  residual  R{  we  have  to  maximize  the  projection.  (ay^ ,  R0  j  At  next  step 
we  must  apply  the  same  procedure  to  Rx . 


^1  ~\ay^  ^1  )ayl  +  ^2 


(5) 
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Residual  of  signal  at  step  n  can  be  written  as  follows: 


Rns  =  Rn-1s-(Rn-1s\ar  V 

\  Yk  /  Yk 


(6) 


Signal  s  is  decomposed  by  set  of  atoms: 


k=0 


(7) 


Algorithm  stops  when  residual  Rn s  of  signal  is  lower  then  acceptable  limit. 

2.2.2. 1  Our  Approach  to  Intrusion  Detection  Algorithm 

In  basic  Matching  Pursuit  algorithm  atoms  are  selected  in  every  step  from  entire  dictionary  which  has  flat 
structure.  In  this  case  algorithm  causes  significant  processor  burden.  In  our  coder  dictionary  with  internal 
structure  was  used. 

Dictionary  is  built  from: 

•  Atoms 

•  Centered  Atoms 

Centered  atoms  groups  such  atoms  from  Dthat  are  as  more  correlated  as  possible  to  each  other.  To 
calculate  measure  of  correlation  between  atoms  function  o(a,b)  can  be  used  [2]. 


(8) 


The  quality  of  centered  atom  can  be  estimated  according  to  : 


(9) 


LPk  ,  is  a  list  of  atoms  grouped  by  centered  atom.  Ok  z  is  mean  of  local  distances  from  centered  atom 
Wc(k  Z)  to  the  atoms  Ac(z)  which  are  strongly  correlated  with  Ac(z) . 

Centroid  Wc(k  Z)  represents  atoms  Ac(i)  which  belongs  to  the  set  i  e  LPk  z  .List  of  atoms  LPk  z  should  be 
selected  according  to  the  Equation  : 


max  o\ 

i&Lpk,i 


(AcarWc<U))^ 


(10) 
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Figure  1 :  Example  atom  from  dictionary 


jl 

l 

In  the  proposed  IDS  solution  ID  real  Gabor  base  function  (Equation  was  used  to  build  dictionary  (11)) 


a  *  ,  (t)  =  c  z  ,a 

u,s,t,(b^  '  u,s,c,(b 


t't  —  us 

V  S  J 


cos(2^(r  -  u) + (f) 


(ll) 


where: 

a{t)  =  -^e~a2  (12) 

V s 

cus  %</>  ~  is  a  normalizing  constant  used  to  achieve  atom  unit  energy, 

In  order  to  create  over  complete  set  of  ID  base  functions  dictionary  D  was  built  by  varying  subsequent 
atom  parameters:  Frequency  %  and  phase  (/)  ,  Position  u  ,  Scale  s  . 

Base  functions  dictionary  D  was  created  with  using  10  different  scales  (dyadic  scales)  and  50  different 
frequencies.  In  Error!  Reference  source  not  found,  example  atoms  from  dictionary  D  are  presented. 


2.2  Experimental  Results 

Percentage  of  the  recognized  anomalies  as  a  function  of  encoded  atoms  from  Dictionary  of  Base  Functions 
is  presented  in  Figure  2.  Five  dictionaries  with  different  parameters  (different  number  of  scales  and 
frequencies)  were  used  in  our  ADS  system. 

Percentage  of  the  recognized  anomalies  for  Dictionary  of  Base  Functions  with  approximately  constant 
number  of  atoms  is  presented  in  Figure  3.  In  this  case  we  try  to  leave  approximately  constant  number  of 
atoms  in  dictionary  but  with  different  proportions  of  scales  and  frequencies. 
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Table  1 :  Matching  Pursuit  Mean  Projection  for  TCP  trace.  Traces  are  analysed  with  the  use  of  20 

minutes  windows 


TCP  trace 

Windowl 

MP-MP 

Window2 

MP-MP 

Window3 

MP-MP 

Mean. 
MP-MP 
for  trace 

Mean  MP- 
MP 

for  normal 

trace 

Mawi  2004.03.06  tcp 

210,34 

172,58 

239,41 

245,01 

240,00 

Mawi  2004.03.13  tcp 

280,01 

214,01 

215,46 

236,33 

240,00 

Mawi  20.03.2004 

(attacked:  worm  Witty) 

322,56 

365,24 

351,66 

346,48 

240,00 

Mawi  25.03.2004 

(attacked:  worm 

Slammer) 

329,17 

485,34 

385,50 

400,00 

240,00 

Table  2:  Matching  Pursuit  Mean  Projection  for  UDP  trace.  Traces  are  analysed  with  the  use  of  20 

minutes  windows 


UDP  trace 

Windowl 

MP-MP 

Window2 

MP-MP 

Window3 

MP-MP 

Mean. 
MP-MP 
for  trace 

Mean  MP- 
MP 

for  normal 

trace 

Mawi  2004.03.06  tcp 

16,06 

13,80 

17,11 

15,65 

16,94 

Mawi  2004.03.13  tcp 

20,28 

17,04 

17,40 

18,24 

16,94 

Mawi  20.03.2004 

(attacked:  worm  Witty) 

38,12 

75,43 

61,78 

58,44 

16,94 

Mawi  25.03.2004 

(attacked:  worm 

Slammer) 

56,13 

51,75 

38,93 

48,93 

16,94 

Table  3:  Matching  Pursuit  Mean  Projection  for  TCP  trace  (traces  consist  of  DDoS  SynFlood 
attacks).  Traces  are  analysed  with  the  use  of  20  minutes  windows 


TCP  trace 

Windowl 

MP-MP 

Window2 

MP-MP 

Window3 

MP-MP 

Mean  MP- 
MP  for  trace 

Mean  MP-MP 
for  normal 

trace 

One  hour  trace 
from  uninal[17] 

1211 

3271 

3007 

2496,333 

860,00 

One  hour  trace 
from  unina2 

[17] 

1906 

1804 

1251 

1653,667 

860,00 

In  Table  1,2, 3, 4  there  are  example  results  taken  from  our  ADS  system.  Traffic  traces  were  analysed  by 
proposed  ADS  with  the  use  of  20  minutes  windows  (most  attacks  (more  than  80%)  last  no  longer  then  20 
minutes).  In  every  window  we  calculate  Matching  Pursuit  Mean  projection  parameter  in  order  to  recognize 
suspicious  traffic  behaviour.  Analysed  traces  are  infected  by  worms  (Tablel,2),  DDos(Table  4)  and  DDoS 
SYNFlood  (Table  3)  attacks. 
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Table  4:  Matching  Pursuit  Mean  Projection  for  TCP  trace  (traces  consist  of  DDoS  attacks). 
Traces  are  analysed  with  the  use  of  20  minutes  windows 


TCP  trace 

Windowl 

MP-MP 

Window2 

MP-MP 

Window3 

MP-MP 

Mean.  MP- 
MP  for 

trace 

Mean  MP- 
MP 

for  normal 

trace 

Backscatter  2008 .11.15 

147,64 

411,78 

356,65 

305,35 

153,66 

Backscatter  2008.08.20 

208,40 

161,28 

153,47 

174,38 

153,66 

Figure  2:  Percentage  of  the  recognized  anomalies  as  a  function  of  encoded  atoms 


Figure  for  -  constant  number  of  atoms  in  Base  Functions  Dictionary 


Figure  3:  Percentage  of  the  recognized  anomalies  for  Dictionary  of  Base  Functions  with 
approximately  constant  number  of  atoms 
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3.0  CONCLUSIONS 

In  this  paper  a  framework  for  recognizing  attacks  and  anomalies  in  the  computer  networks  is  presented. 
Our  methodology  is  based  on  both  statistical  and  signal  based  features.  The  major  contribution  and 
innovation  is  the  application  of  Matching  Pursuit  algorithm  to  calculate  network  traffic  features.  The 
effectiveness  of  the  proposed  approach  has  been  proved  in  attack  and  anomaly  detection  scenarios.  Our 
framework  can  be  applied  to  enhance  military  networks  since  it  uses  signal -based  features.  Such  features 
can  be  calculated  for  encrypted  traffic  since  flow  characteristics  are  extracted  without  considering  the 
payload.  Future  work  focuses  on  algorithms  optimization  so  that  our  framework  can  be  applied  to  real¬ 
time  network  security  enhancement. 
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